Hierarchical Sparse Spectral Clustering For Image Set Classification

We present a structural matching technique for robust classification based on image sets. In set based classification, a probe set is matched with a number of gallery sets and assigned the label of the most similar set. We represent each image set by a sparse dictionary and compute a similarity matrix by matching all the dictionary atoms of the gallery and probe sets. The similarity matrix comprises the sparse coding coefficients and forms a fully connected directed graph. The nodes of the graph are the dictionary atoms and the edges are the sparse coefficients. The graph is converted to an undirected graph with positive edge weights and spectral clustering is used to cut the graph into two balanced partitions using the normalized cut algorithm. This process is repeated until the graph reduces to critical and non-critical partitions. A critical partition contains atoms with the same gallery label along with one or more probe atoms whereas a non-critical partition either consists of only probe atoms or atoms with multiple gallery labels with no probe atom. Using the critical partitions, we define a novel set based similarity measure and assign the probe set the label of the gallery set with maximum similarity. The proposed algorithm is applied to image set based face recognition using two standard databases. Comparison with existing techniques shows the validity and robustness of our algorithm in the presence of outlier images. A schematic diagram of the proposed algorithm is shown in Fig. 1. The intrinsic data dimensionality in the image sets is often less than the apparent dimensions. Therefore, we reduce the data dimensionality using PCA basis computed from the training (gallery) sets. For each reduced dimensionality gallery set, we pre-compute sparse dictionaries of varying sizes. A sparse dictionary must be able to represent all images in an image set as a sparse linear combination of its atoms. Given an image set Xi = {x j}i j=1 ∈R m×ni , its dictionary Di ∈Rm×pi should be able to minimize a cost function n ∑ ni j=1 f (x j,Di). Each column of Di represents a basis vector for the image set Xi. We use the convex `1 formulation of the Lasso as the cost function [3]